We deal with a lot of fake controversies here, or “manufactroversies”, ones that are ginned up for political purposes, without a genuine underlying scientific controversy (like the safety of vaccines, for example). However, we also deal with some legitimate scientific controversies. These result when the existing scientific evidence is ambiguous enough that different bottom-line interpretations are reasonably defensible.
So here is another one – based on current evidence, what should our recommendations be regarding the consumption of red and processed meat? The evidence has been building over the last two decades that there is some possible risk from red (meat from a mammal) and processed (smoked, cured, salted) meat with regards to cardiovascular risk, cancer risk, and overall mortality. However, as we have discussed here previously, the evidence has always been weak.
This controversy has been flamed by a new study published in The Annals of Internal Medicine. The study is actually five parallel systematic reviews, four looking at health outcomes of eating red and processed meat, and one looking at people’s values regarding meat consumption. The paper is the result of a consortium put together for this purpose, with careful vetting to minimize conflicts of interests, and with a 14-person panel voting on final recommendations. They write:
These recommendations are, however, primarily based on observational studies that are at high risk for confounding and thus are limited in establishing causal inferences, nor do they report the absolute magnitude of any possible effects. Furthermore, the organizations that produce guidelines did not conduct or access rigorous systematic reviews of the evidence, were limited in addressing conflicts of interest, and did not explicitly address population values and preferences, raising questions regarding adherence to guideline standards for trustworthiness (5–9).
A potential solution to the limitations of contemporary nutrition guidelines is for an independent group with clinical and nutritional content expertise and skilled in the methodology of systematic reviews and practice guidelines, methods that include careful management of conflicts of interest, to produce trustworthy recommendations based on the values and preferences of guideline users. We developed the Nutritional Recommendations (NutriRECS) (7) international consortium to produce rigorous evidence-based nutritional recommendations adhering to trustworthiness standards (10–12).
Based on their own reviews, they make the “weak” recommendation based on low to very low quality evidence (with a consensus of 11/14 of the panel) that people not change their current eating habits with respect to red and process meat. This is a different bottom-line interpretation of the same evidence as previous nutritional recommendations, which generally recommend reducing red and processed meat consumption. As a side note, the authors state that these recommendations are based entirely on health data, without consideration of environmental effects or ethical considerations regarding animal welfare.
The authors attribute the difference in recommendation to several factors. In general, they credit the thoroughness of their own review, with explicit consideration of the quality of evidence, conflicts of interest, and effect sizes. But here are the specific factors:
- They weight randomized controlled trials higher than observational studies.
- They give great consideration to the limited quality of the evidence overall.
- They are not impressed with the small absolute effect sizes and the limited statistically significant outcome measures.
- They err on the side of not recommending any change based on weak evidence, rather than asked people to change their lifestyle given the level of uncertainty.
- They specifically consider the value system of people regarding eating meat and judge that, if people were fully informed regarding the absolute risk reduction and quality of the evidence, they would not think it is worth changing their eating habits.
For example, regarding unprocessed red meat, here is their summary of the evidence of harm:
Dose–response meta-analysis results from 23 cohort studies with 1.4 million participants provided low- to very low-certainty evidence that decreasing unprocessed red meat intake may result in a very small reduction in the risk for major cardiovascular outcomes (cardiovascular disease, stroke, and myocardial infarction) and type 2 diabetes (range, 1 fewer to 6 fewer events per 1000 persons with a decrease of 3 servings/wk), with no statistically significant differences in 2 additional outcomes (all-cause mortality and cardiovascular mortality) (16). Dose–response meta-analysis results from 17 cohorts with 2.2 million participants provided low-certainty evidence that decreasing unprocessed red meat intake may result in a very small reduction of overall lifetime cancer mortality (7 fewer events per 1000 persons with a decrease of 3 servings/wk), with no statistically significant differences for 8 additional cancer outcomes (prostate cancer mortality and the incidence of overall, breast, colorectal, esophageal, gastric, pancreatic, and prostate cancer) (13).
The risk reduction is very small, without any significant change in all-cause mortality.
It’s also important to note why the evidence is of low to very low quality. Most of the data is observational. A 2009 study nicely summarizes the confounding factors:
Subjects who consumed more red meat tended to be married, more likely to be of non-Hispanic white ethnicity, more likely to be a current smoker, have a higher body mass index, and a higher daily intake of energy, total fat and saturated fat; whereas they tended to have a lower education level, were less physically active and consumed less fruits, vegetables, fiber and vitamin supplements.
Essentially the confounding factors are noise, and the tiny risk reduction is the signal. So what they are essentially saying is that the signal to noise ratio is extremely low, so we can’t be certain of the ultimate effects. So really the conclusion of the authors is not that there is no health risk from red or processed meat. Their conclusion is that we don’t really know. The quality of the evidence is simply too weak to form any confident recommendations. But part of that low confidence is likely the fact that the absolute risk is probably low. If there were a substantial risk, we would be seeing it more confidently in the data (again – signal to noise).
The rest is a matter of interpretation and judgement. Based on weak evidence of a small effect size, do we tell people to change their eating habits? These authors decided no, while other experts erred on the side of caution to recommend limiting red and processed meat.
There are two things that everyone seems to agree on – we need better quality data, and we are unlikely to get it. We simply cannot do the large, long term, randomized controlled trial that would settle the debate. So we must be content with small controlled trials or large uncontrolled observational studies. If the true effect size is on the low side, however, this type of data is unlikely to ever definitely settle the controversy.
My personal bottom line is – look at the data and do what you want (you’re going to do that anyway), based on your own values. I will add one wrinkle to the evaluation – there is good evidence that it is important to health to eat plenty of vegetables. Caloric intake is a zero sum game, so if you eat a lot of meat you are probably not eating enough plants. The better recommendation, therefore, may not be to avoid meat, but to make sure you get enough variety in your diet overall, including plenty of fruits and vegetables.
In fact, I think that a lot of nutritional science can be boiled down to simply eating a varied diet. All the risks and nutritional needs will likely average out. It’s also a very simple and easily understood recommendation (like many science-based health recommendations) – have a varied diet. Done.
There is another layer to this controversy however, the meta-layer of the quality of nutritional science itself. As the NY Times reports:
In a statement, scientists at Harvard warned that the conclusions “harm the credibility of nutrition science and erode public trust in scientific research.”
That may be a defensive interpretation, but I acknowledge this too is complicated. I wrote about this recently (unrelated to the current topic). What we are advocating as scientific skeptics is a nuanced approach to science. Science is complicated, has lots of flaws and biases, and takes a long time to work out. A lot of evidence is weak and flawed, and we need to be humble before jumping to premature conclusions. But we can arrive at confident conclusions eventually, if we follow a careful and rigorous process.
This is neither nihilistic nor naive – but optimistic and realistic. Pointing out the complexity of nutritional research, in my opinion, will only erode a naïve trust in science, not an informed skeptical trust. That naïve trust is inherently fragile, and not worth protecting. I also think this study can foster a healthy and completely appropriate skepticism toward the current state of nutritional science. This is not to denigrate the field, but to point out how difficult it is to do good clinical nutritional science out there in the real world.
Much of our nutritional science is toward the weak end of the spectrum, because of the inherent limits of observational data and the difficulty of controlling experimentally what people eat. We have to acknowledge that and be open about these limitations. Pretending otherwise is a misguided strategy and will backfire. Hopefully this healthy skepticism will foster more mature and informed attitudes among the public, and higher quality research.
In the end this paper can be a great thing for the field, if we take away the proper lessons.